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THE  METHOD  OF  CONTRACTING  ELLIPSOIDS 
by 

James  E.  Falk 

1.  Introduction 

The  October  6,  1979  issue  of  Science  News  [v*  ]  featured  a  cover 
story  entitled  ‘'Linear  Programming:  Solid  New  Algorithm,"  which  an¬ 
nounced  the  discovery  of  a  "totally  new  algorithm  for  linear  program¬ 
ming"  by  the  Soviet  mathematician  L.  G.  Khachian.  Science  [6]  devoted 
a  page  and  a  half  of  their  November  2,  1979  edition  to  the  event, 
hinting  that  even  the  difficult  traveling  salesman  problem  might  now 
be  rendered  tractable. 

The  New  York  Times  took  over,  in  articles  on  November  7,  11,  and 
27,  1979,  amplifying  the  potential  applications  of  this  "surprise  dis¬ 
covery  by  an  obscure  Soviet  mathematician." 

The  genesis  of  all  this  publicity  was  an  article  by  L.  G. 

Khachian  appearing  in  the  January-February  1979  issue  of  Doklady  en¬ 
titled  "A  Polynomial  Algorithm  in  Linear  Programming"  [4 ] .  English 
translations  of  the  Russian  version  appeared  later.  Hungarian  mathe¬ 
maticians  Gacs  and  Lovasz,  then  at  Stanford,  distributed  a  set  of  their 
notes  on  Khachian's  paper,  which  included  some  variations,  comments,  and 
proofs  of  Khachian's  results  [2]. 

The  purpose  of  this  paper  is  to  set  down  a  detailed  description 
of  the  algorithm.  Including  proofs  not  available  in  the  aforementioned 
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sources.  The  title,  ,rThe  Method  of  Contracting  Ellipsoids,*'  is  chosen 
to  accurately  reflect  the  nature  of  the  algorithm.  It  is  used  in  pref¬ 
erence  to  the  currently  common  description,  "Khachian's  Method,"  since 
the  algorithm  itself  has  roots  in  literature  predating  Khachian's  paper 
(as  he  himself  points  out)  in  articles  published  by  N.  Z.  Shor  [9],  [10], 
D.  B.  Yudin  and  A.  S.  Nemirovskii  [13],  and  A.  Yu.  Levin  [7].  As  we  will 
point  out,  Khachian's  (very  significant)  contribution  was  to  establish 
the  polynomial  property  of  the  method  when  applied  to  a  system  of  strict 
linear  inequalities. 


t: 


2.  Geometrical  Description  of  the  Method 

Consider  a  system  of  linear  inequalities  (see  Figure  1)  describ¬ 


ing  a  set  F  CRn  enclosed  in  an  ellipsoid  with  center  .  It 


is  desired  to  locate  a  point  x  £  F  .  If  x  £  F  ,  we  are  done;  other¬ 


wise  there  is  some  linear  inequality  ax  <  3  which  is  violated  by  xU  , 


i . e . ,  ax  >  3  . 


Accb^cion  for  ' . . 

IVC  TAB  H 

U:ic:incu;ccd  H 

Jujt  il'ic..tion 


on 


Bv 


Figure  1. — Contracting  ellipsoids. 
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Then  the  set  F  Ce\  where  E^  is  an  ellipsoid  of  minimal 


volume  containing  the  set  H  (x:  ax  <  ax^}  .  The  new  ellipsoid 

-  2  - 
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has  center  »  which  is  then  tested  for  feasibility  to  F  ,  and  the 
method  continues  by  replacing  and  by  and  x^  . 

0  1 

Thus  a  sequence  E  ,E  ,...  is  generated.  We  will  show  that 

k 

the  volumes  v(E  )  decrease  geometrically  by  a  factor  which  is  a  func¬ 
tion  of  the  dimensionality  n  of  the  space  Rn  .  Assuming  F  is  not 
empty,  the  ellipsoids  will  tighten  up  in  volume  around  S  ,  so  that 
eventually  an  ellipsoid  will  be  produced  with  center  in  F  * 

3.  Derivation  of  the  Method 

In  this  section  we  will  derive  the  updating  formulas  for  the 
Method  of  Contracting  Ellipsoids.  The  algorithm  is  summarized  below. 

3.1  The  Method  of  Contracting  Ellipsoids 

We  wish  to  solve  (or  establish  the  insolvability  of)  the  system 
Ax  <  b  ,  where  A  is  m  *  n  and  b  is  m  x  1  . 

Start:  Locate  x  £  Rn  and  a  radius  r  >  0  such  that  the 
sphere  {x:  ||x  -  x||  <  r}  contains  a  feasible  point  (if  any).  Set 
J  *  rl  . 


Step:  Test  the  point  x  for  feasibility, 

feasible,  identify  a  constraint  ax  <  3  such  that 


If  x  is  not 
ax  >  6  . 


(i)  Set 


6 


ax  -  3 

TI^TT  * 


(i) 


*  + 

If  o  >  1  ,  there  is  no  solution.  If  6*1,  x  below 

is  the  only  solution,  provided  it  is  feasible. 

(nS+l\  JTJa 
\  n+1  /  |  ]  Ja  f|  ' 
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(ii) 


X 


X  - 


(2) 
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(iii) 


I+  ■  “ferj1 


Ja(Ja) 

l  |j»I  l2 J 


(3) 


These  formulas  differ  from  those  presented  by  Shor  [10]  and  Kha- 
chian,  primarily  by  the  inclusion  of  the  factor  <5  above.  This  factor 
is  related  to  a  deeper  cut  than  the  aforementioned  authors  report  (they 
take  6  -  0  ),  and  leads  to  faster  convergence.  The  formulas  above  were 
presented  by  Wolfe  [12],  and  will  be  derived  herein. 


In  the  material  which  follows*  B  denotes  a  symmetric,  positive 
definite  matrix  with  eigenvalues  X^t...fX  *  Thus» 

B  =  PTDP  ,  (4) 

where  P  is  an  orthogonal  transformation,  and  D  =  diag{X^, . . . , X^}  . 
Setting 

J  '  DT>  ,  (5) 

U  U  Xr 

where  D2  -  diagiX 2, . . . , X 2}  ,  we  have  an  equivalent  factorization  of 
1  n 

B  as 

B  =  JTJ  .  (6) 

Given  B  and  a  vector  x  ,  E(B,x)  ,  or,  more  simply  E  ,  will 
denote  the  ellipsoid 

E  *  E(B,x)  *  {x:  (x-x)T  B  ^(x-x)  <  1}  . 


Lemma  1:  The  ellipsoid  E  is  an  affine  transformation  T  of 

2 

the  unit  sphere  S  =  { y :  ||y||  <  1}  .  Specifically, 

T (S)  =  E  , 

—  T 

where  T(y)  =  x  +  J  y  . 


—  T  -1  — 

(x-x)  B  (x-x)  <  1  is  equivalent  to 


—  T  -l  -1  T  — 

(x-x)  J  (J  )  (x-x)  <  1  . 
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Setting  y  38  ( J *  )  (x-x)  establishes  the  result. 

Figure  2  illustrates  this  lemma.  Note  that  J  is  nonsingular 
from  (5).  Lemma  1  will  be  used  extensively,  since  it  is  easier  to  work 
with  spheres  than  ellipsoids. 


Figure  2. — Ellipsoids  are  affine  transformations  of  spheres. 


Lemma  2:  The  volume  of  E  is 


where  v(s)  is  the  volume  of  the  n  dimensional  sphere. 

Proof :  From  the  previous  lemma,  E  is  image  of  the  unit 

—  T 

sphere  S  under  the  transformation  T(y)  *  x  +  J  y  .  Therefore, 

v(E)  =  | det(JT) |  v (s) 

■  |det  Dz\  |det  P|  v(s) 
n  k 

m  n  v(s)  , 
j*i  J 

u 

since  D  is  diagonal  and  P  is  orthogonal. 
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This  lemma  will  prove  useful  in  two  places,  the  first  in  estab¬ 
lishing  the  formulas  for  a  minimal  volume  ellipsoid  encapsulating  a 
given  region,  and  also  in  computing  the  decrease  in  successive  volumes 
as  the  algorithm  progresses. 

Turning  attention  to  the  sphere,  we  define  a  "truncated  sphere" 


S(e  ,6)  as 


S(e1,<5)  =  {z:  ||z||  <  1,  >  5}  , 


1  T 

where  e  *  (1,0,..., 0)  and  0  5  <  1  .  Figure  3  illustrates  this 

definition.  We  wish  to  encapsulate  S(e\($)  in  an  ellipsoid  E  of 
minimal  volume. 


S(e\6) 


o,/a 


Figure  3. — A  truncated  sphere. 


We  will  restrict  our  attention  to  candidate  ellipsoids  of  the 

^  ^  —  2  2  2 
form  E(B,z)  =  (z:  T  (z  -£)  +  ^0z9  +  ...  +  T  z  <  l)  ,  which  contain 

11  2  z  n  n  * 

sn{z:  z^  =  6}  and  on  . 

~  -  2  2 

Lemma  3:  The  ellipsoid  E(B,z)  =  +  T2Z2  +  •  ••  + 

2 

1  z  <ll  of  minimal  volume  which  contains  the  surface  of  the  sphere 
n  n  r 

{z:  ||z||  ®  1,  =  6}  and  the  point  e^  has 

/  n6+l  n  n\T 

2  =  \  n+l  ’  °*  °) 


z 
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and 


8  =  diag 


/ n(l-6) \2  / n2(l-62)\  /n2(l-62)\ 

1  "+1  j  ’  V  n2-l  r  ""  V  n2-l  ) 


Proof:  We  will  first  characterize  the  values  of  T, 

1  n 

and  £  ,  which  yield  a  minimal  volume  ellipsoid  containing  only  the 
points  e^  and  the  n-1  points  (6,  /l -6" ,  0,  .  0), 

(6,  0,  . 0,  /l“62)  .  We  will  then  show  that  the  resulting  E  con¬ 
tains  all  of  S(e^,6)  . 


Since  e  c  9E  ,  we  have 


Tjd-er  =  i  , 


while  (6,  0,  . 0,  /l-6  2 ,  0,  0)  e  3E  implies 


T^fi-S)2  +  Tk(l-62)  =  l  ,  for  k=2 , . . .  ,n  ; 


i.e.  , 


and 


r  = 

1 


(l  O 


T  =  a-oJLz  k=2 

a-6  m-o 


Thus,  using  (7),  we  have 


v(E)  =  1(1-0 


(K 


vn-1 


(1-62)(1-Q2 


(1-5)2  -  (6-02 


v(s)  . 


It  is  easier  to  work  with  v  (E)  ,  and  since  this  is  minimized  at  the 
same  value  of  £  as  is  v(E)  ,  we  lose  no  generality.  Thus, 


V2(E)  =  (l-O2 


(l-62)(l-£;)2 

Ld-O2  -  (6-C)2J 


n-1 


V2(s) 


.a-Q2n(i-62.)"~1  v2(s) 

[  (l-€)  -  (6-0  ] 


7  - 
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This  quantity  is  minimized  when  its  log  is  minimized,  so,  taking  the 

2  - 

derivative  of  the  log  of  v  (E)  ,  and  setting  it  to  zero,  we  obtain 


as  the  (unique)  point  which  satisfies  the  resulting  equation.  A  check 

2  - 

of  the  second  derivative  of  the  log  of  v  (E)  establishes  that  the 
above  £  minimizes. 

We  thus  have 


= 

1  \n(l-5)  / 

.±-i) 


T,  =  - ,  k=2  , .  .  .  ,n 

(1-5“) 

Equations  (8)  and  (9)  define  the  minimum  volume  elixpsoid  containing 

(1,0,..., 0)  and  the  (n-1)  points  of  the  form  (5,  0,  ...,  0,  /l-67 , 

0,  .  .  .  ,  0)  . 

Finally,  we  must  show  that  all  of  S(e\fi)  is  contained  in  E  . 


so  that 


Noting  that 


we  have 


n2-l 

n2(l-62) 


(«  T,  »  . ■  T  )  , 


B  =  diag{x1,  T,  . . . ,  t) 


t  _  T  .  2(n+l) (6n+l) 

1  2  2  ’ 
A  n  (1-6)  (1-6  ) 
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(  z-7. )  TB~ 1  (z-z  ) 


T--1  - _ 1  —T  v  - 1 

=  z  B  z  -  2zB  z  +  z  B  z 

.1  |  1 2  2  (n+1)  (6  n+1)  2  _  -  ^  -  ,2 

r  1  I  z  II  +  o  2  zi  ^xzz  -t-  T(z) 

n  u-s)(i-<n  1  111  11 


■t 


n  -1\/  1 


n2  /V 1-62J 


I  2  2(n+l)(6n+l)  2 

I  +  -a - 2“  zi 

n  (1-6) (1-6^)  1 


-  ■fffiSW-.  •  »>'  ■ 


2  2 

Now,  adding  and  subtracting  coefficients  of  ||z||  ,  ,  and 


Zj  ,  we  obtain 


(z-z)TR  ^ (z-z) 


V  n  /\l-6  /  „(!-«)< l-«>  1  1 


+  2 


+  2 


(n+1) (6n+l)\/  1 

n“(l-6)" 


f 

/(n+1) (6n+l)\/  1 

V  n2(l-6)2  Jll+6_ 


ijuri) 


But  the  first  three  terms  are  less  than  or  equal  to  zero,  while  the  last 
three  terms  sum  to 


2 (n+1 ) (6n+l ) (-6)  +  (n2-l)(l-6)  +  (6n+l)2(l+6) 
n2(l-6)2(l+6) 


so  that 


i . e.  , 


(z-z)TB  ^(z-z)  <  1  , 
S (e1 , 6)  C  E  , 


and  the  proof  is  complete. 


-  9  - 
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Having  now  worked  out  the  formulas  for  enclosing  a  truncated 
sphere,  the  next  step  is  to  translate  our  results  to  apply  to  a  trun¬ 
cated  el L ipso id. 

Suppose  we  have  an  ellipsoid  E  =  E(B,x)  ,  and  suppose  a*x  >  6  . 
We  shalL  assume  E  contains  points  in  E  satisfying  a*x  <  3  .  As 

e^  was  the  point  of  tangency  of  the  hyperplane  (z:  z^  *  1}  to  the 

sphere  in  z-space,  we  wish  to  locate  the  point  t  which  is  the  point  of 
tangency  of  the  hyperplane  {x:  a*x  -  a*t}  and  the  ellipsoid  E  (see 
F igure  4 ) . 


ax  <  3 


Figure  4. — Definition  of  t  . 


Lemma  4:  Given  an  ellipsoid  E(B,x)  and  a  half  space 
(x:  a*x  <  P)  ,  the  point  t  which  lies  in  this  half  space  and  is  tan¬ 
gent  to  c)E  and  the  hyperplane  (x:  a*x  =  a*t}  is 


■  11  -  (tiAtt)  jTja  • 


Proof :  t  solves  the  problem 


minimize  a#x 


—  T  -1  — 

subject  to  (x-x)  B  (x-x)  >  1  . 

This  is  a  convex  program  with  feasible  region  having  a  nonempty  interior, 
so  the  first  order  optimality  conditions  are  necessary  and  sufficient. 


Thus  there  is  a  u  >  0  such  that 


-  10  - 
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z-space,  with  t  *>  e  ,  and  {x:  a*x  <  3)  ■>  {z:  z^  >  6}  ,  where  <5  = 


tjjf|T  *  (See  Fi8ure  5-) 


Figure  5. — Transforming  t  into  . 


Proof :  The  first  transformat  ion  was  derived  in  Lemma  1,  The 

image  of  t  under  this  transformation  is 


q  =  (JT)  L(t-x) 


We  now  wish  to  construct  Q  so  that 

-  1 

Qq  *  e  . 

Since  Q  will  be  orthogonal,  and  since  ||q||  =  1  ,  we  may  set  its 
first  row 


—TIT 

«>1  ’  ?  -  -  ttAtt  (Ja)  ■ 


mutually  orthogonal,  have  norm  1,  and  be  orthogonal  to  (Q) .  . 


•<*£.>?»  SB* 
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one  of  the  constraints  a*x  <  P  of  the  system  Ax  <  b  is  violated  by 
x  .  We  then  surround  the  set  E  0  (x:  a*x  <  SI  in  an  ellipsoid 

E  =  E  (B  ,x  )  of  minimal  volume.  As  E  must  contain  a  point  feasi¬ 
ble  to  F  ,  and,  as  we  shall  show,  E+  has  a  smaller  volume  than  E  ; 
it  is  more  likely  that  x+  is  feasible  to  F  . 


All  that  remains  is  to  derive  the  proper  formulas  for  B  and 


Theorem  1:  Assume  F  C  E(B,x)  .  Then  F  C  E+(B+,x+)  ,  where 
+  + 

x  and  B  are  given  by 


(i)  6  = 


a*x  -  3 

W  ’ 


+  —  ( no+l \  J  Ja 

(ll)  x  =  x  -  hsr)  tjjT[t  • 

;m)  j+  ■  M  -  (■  -  (SMfl 


Ja(Ja) 


—  +  +  x  4. 

where  a«x  <  P  is  violated  by  x  and  B  =*  (J  )  J 


Proof :  Using  the  transformations 


y  =  (dT)  I(x-x) 

z  =  Qy  * 


where  QTQ  =  I  and  (Q)  ^  yyj  (Ja)T  ,  we  map  the  set 

E  O  {x:  a*x  <  P)  into  the  set  {z:  z^  >  6}  (Lemma  5).  Clearly  this 
set  contains  the  image  of  a  point  feasible  to  F  . 


In  z-space,  the  ellipsoid  E(B,z)  ,  where 


-Ntf- . M 
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z  -  (nn+1  ,  0,  ....  o) 


contains  (z:  z^  >6}  »  and  is  of  minimal  volume  (Lemma  3).  All  that 

remains  is  to  transform  E  back  into  x-space,  using  the  transformation 
of  Lemma  L,  and  the  Q  of  Lemma  5. 

Working  on  x+  first,  we  have 

y  =  Q  z 


‘  (t?t)  (-  TTTaTr)  Ja 


from  Lemma  5.  Then 


+  -  ^  J  + 

x  =  x  +  J  y 


nS+l\  J  Ja 

n+l  )  fl  Ja  n  ’ 


which  is  formula  ( ii )  . 


Now  working  on  B  ,  we  have 


if  and  only  if 


if  and  only  if 


(z-z)T(B)'1(z-z)  <  1 


(y-y)V(B)"1Q(y-y)  <  l 


(x-x)V1qT(B)'1Q(J  1)t(x-x)  <  l  ; 


»+  =  [j“1QT(b)"1Q(J~1)T] 

.T_T  m  \(  n+l  \2  VLLQdLl  n2 (1-62) ) 

-  JQ  diag^n(1_6)J  ,  ^2_1  ’  n2  x  |  QJ  • 


1 
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Now 


..  S(  n+l  \2  n2(l-<$2)  n2(l-62)) 


so  that 


s+ '  (ixir) jT,T  t1 '  diagi1 '  (<w(^))-  ° . °0qj 


But  since 


JV  diag{t , 0, . .  .  ,0>QJ  =  /(Q^tOQ  J  , 
where  is  the  first  row  of  Q  ,  we  obtain 

JTQT  diag{t,0 . 0}QJ  =  JT  (-  jjj-|  j-^Ja(t)  (-  f|  | ) ^ jT J 


(iMI2) 


(Ba) (Ba)  ; 


K 


i. e.  , 


/  /<l-b)(n-l)^ 

1  (Ba) (Ba)T 

r  \(l+b)(n+l )fj 

IMI2  . 

It  is  easily  checked  that  the  J  of  formula  (iii)  gives 


(J+)V  =  B+  , 


as  above,  so  we  are  done. 


4 .  Properties  of  the  Method 

Having  derived  the  method,  we  now  turn  to  some  of  its  properties. 
One  of  the  most  useful  observations  is  that  of  the  decrease  in  the  vol¬ 
ume  of  the  encapsulating  ellipsoids. 
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Theorem  2;  At  each  iteration  of  the  method,  the  volume  of  the 
encapsulating  ellipsoid  decreases  by  the  factor 


Proof :  Returning  to  Lemma  5,  we  note  that 

E  =  Z(E+)  and  S  =  Z(E)  , 

T  -1 

where  Z  Is  the  composite  transformation  z  =  Q(J  )  (x-x)  ,  and  S  is 

the  unit  sphere.  Since  affine  transformations  do  not  change  the  propor¬ 
tion  of  the  volume,  we  have 

v(E+)  _  v(E) 
v(E)  v(S)  * 

But 

n^l 

2  ' 

v(S)  , 


v(E)  = 


n(l-<S) 

(n-1) 


n2(l-62) 


n2-l 


SO 


v(E+) 

v(E) 


n-1 


(1-6) (l-62 )  2 


<*<£)' 


n-1 

2" 


(15) 


Corollary :  v(E 


S+)  <  v(E)  •  (e  2(n+1))(l-6)(l-62)  2  . 


Proof : 
part icular , 


Since  e *  is  a  convex  function,  1 +  t  <  et  .  In 


and 


n 

n+1 


1  "  n+1  <  e 


1 

n+1 


n2-l 


1  n2-l 

1  +  — x —  <  e 

n-1 
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It  should  be  noted  here  that  the  factor 


n-1 


is  close  to  1  for  large  n  ,  producing  relatively  little  decrease  in 

(  2\ 0-1/2 

successive  volumes.  The  factor  (1-6) \l-6  /  ,  however,  is  domi¬ 

nated  by 


-6-6 


W) 


which,  for  6  >  0  „  is  small  for  large  n  ,  resulting  in  significant  vol¬ 
ume  decrease.  Recall  6  is  a  measure  of  the  infeasibility  of  the  cur¬ 
rent  estimate.  If  6  is  determined  by  the  formula 


the  largest  decrease  in  volume  per  iteration  is  obtained.  Note  6-0 
implies  that  the  current  point  is  feasible,  whilst  6=1  implies  that 
the  next  point  will  be  feasible.  The  next  table  illustrates  the  joint 
effects  of  6  and  n  on  expression  (15). 


5 .  Polyn omial  Boundedness  of  the  Method 

The  most  startling  property  of  this  method,  and  Khatchian's  main 
contribution,  is  that  the  amount  of  work  required  to  execute  the  tech¬ 
nique  is  bounded  by  a  polynomial  function  of  the  problem  data.  By  con¬ 
trast,  the  popular  Simplex  Method  is  not  polynomially  bounded,  as  Klee 
and  Minty  [5l  pointed  out  in  1972. 
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5. 1  The  Klee-Minty  example 

With  the  usual  pivot  selection  rule,  the  problem 

maximize  J  10^n  ^ x . 

j=l  J 

subject  to  x.  +  2  £  10^  ^x.  <  102i  2  ,  1=1,..., n 

1  j<i  3 

x .  >  0 
J  ” 

,  ~n— 1  . 

takes  2  pivot  steps  to  execute. 

Figure  6  (not  to  scale)  illustrates  the  sequence  of  vertices 
visited  by  the  Simplex  Method. 

This  example  is,  of  course,  only  valid  under  the  assumption  that 
the  usual  pivot  selection  rule  is  employed.  Jeroslov  r3],  however,  has 
shown  that  similar  examples  can  be  constructed  to  illustrate  that  an 
exponential  amount  of  work  may  be  required  under  any  entering  variable 
selection  rule. 

It  should  be  emphasized  that  the  aforementioned  examples  only 
show  that  the  worst  case  could  occur.  Over  40  years  of  experience  with 
the  Simplex  Method  have  shown  that  such  worst  cases  seldom,  if  ever, 
occur  in  practice,  and,  indeed,  the  method  has  proven  to  be  remarkably 
efficient  for  a  tremendously  wide  variety  of  problems. 

The  proof  of  the  polynomial  boundedness  property  of  the  Method 
of  Contracting  Ellipsoids  was  first  established  by  Khachian  [4]  and 
later  refined  by  Gacs  and  Lovasz  (2].  Since  their  papers  contain  a  very 
detailed  proof  of  the  polynomial  boundedness  property,  we  shall  not 
repeat  it  here,  but  rather  sketch  the  reasoning  behind  it. 

Suppose  we  wish  to  solve  (or  determine  infeasibility  thereof) 
the  system  of  strict  inequalities 

Ax  <  b  ,  (16) 
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where  the  entries  of  A  and  b  are  integers.  Define  the  constant 


L  = 


m  n  m 

},  1  1°8?(  |af  J  +  O  +  I  log9(|b  |  +  1)  +  log  n 

Li=i  j=i  11  i=i  “  1  1 


+  l  . 


The  following  lemma  is  used  in  the  proofs  of  the  theorems  which 


follow. 


Lemma  6:  Let  B  be  a  square,  nonsingular  submatrix  of  the 
matrix  [ A  j  b ]  .  Then 

| det  B|  <  2L/n  . 

The  proof  of  this  lemma  is  a  result  of  the  fact  that  |det  B| 
is  bounded  above  by  the  product  of  the  norms  of  the  rows  of  B  .  This 

T 

fact  follows  by  reducing  BB  to  Its  diagonal  form,  and  applying  the 

T 

arithmetic-geometric  inequality  to  B.B^  >  w^re  B^  is  the  i th  row  of 
B  .)  The  product  of  the  norms  of  the  rows  of  B  is  easily  shown  to 
be  bounded  by  2^/n  . 

This  lemma  is  used  in  the  proof  of  the  following  pair  of  theo¬ 
rems.  The  first  theorem  establishes  an  upper  bound  on  norms  of  any  so¬ 
lutions  of  (16),  while  the  second  establishes  a  lower  bound  on  the  vol¬ 
ume  of  the  set  of  all  solutions. 


Theorem  3:  Any  vertex  v  of  the  system 


sat isf ics 


Ax  <  b 

|Vj |  <  2L/n  . 


The  proof  of  this  theorem  simply  looks  at  v.  as  expressed  by 

Cramer?s  rule,  as  the  quotient  of  two  determinants.  The  previous  lemma 
yields  an  upper  bound  on  the  numerator  of  this  quotient,  while  the  de¬ 
nominator  is  at  least  one  in  absolute  value,  since  the  elements  of  the 
determinant  are  integers. 
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Theorem  4:  If  the  system 


Ax  <  b 

has  a  solution,  then  the  volume  of  its  interior  inside  the  ball 
(x:  I  lx  I  I  <  2^}  is  at  least  2  . 


(16) 


The  proof  of  this  theorem  begins  by  establishing  the  existence  of 
a  nondegenerate  simplex  generated  by  vertices  v^,..,,vn  of  the  system 


Ax  <  b  . 

Thus,  the  volume  of  the  solution  set  described  by  (16)  is  bounded  below 
by  the  volume  of  the  simplex,  which  is  the  absolute  value  of 


But  this  is  shown  to  be  bounded  below  by 


9 


-nL 


The  following  theorem  establishes  the  polynomial  boundedness  of 
the  method  for  the  system  (16) . 


Theorem  5 :  If  (16)  has  a  solution,  then  the  Method  of  Contract¬ 
ing  Ellipsoids  will  locate  a  solution  within  [2n(n+l) (2L+1) ]  steps,  if 

one  starts  with  an  ellipsoid  =  (x:  ||x||  <  2^}  . 

By  Theorem  4,  if  (16)  has  a  solution,  then  the  set  of  all  solu¬ 
tions  has  volume  at  least  2  nL  .  By  Theorem  5,  an  initial  ball  of 

radius  2^  will  contain  the  solution  of  (16),  if  any.  With  6  set  to 
0  (the  most  conservative  of  settings)  the  corollary  to  Theorem  2  yields 

a  per  step  decrease  of  an  amount  e  f2(n+l)/l]  ^  Therefore,  after  k 
steps,  the  volume  of  the  enclosing  ellipsoid  is 

k 

,_kN  *2(n+l) 
v(E  )  =  e  7  v(E  )  . 

We  wish  to  determine  a  value  of  k  such  that  the  volume  of  the  enclos¬ 
ing  ellipsoid  is  less  than  the  volume  of  the  solution  space. 
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Noting  that  the  volume  of  the  initial  sphere  is  less  than 
(2  x  2^)n  ,  the  volume  of  the  smallest  enclosing  box  whose  sides  are  of 
length  2  x  2^  ,  it  suffices  to  determine  an  integer  k  such  that 

_ _ k _ 

e  2(n+l)  #  2n(L+l)  <  2-nL 

i. e.  , 

-~7~V  +  n(L+l)  J£n2  <  -nL  SLn2  . 

2 (n+1) 

A  value  of  k  which  satisfies  this  is 

k  *  [2n(n+l)(2L+l)]  . 

Thus,  if  no  feasible  point  is  found  after  k  iterations,  there  cannot 
be  a  feasible  point. 

Note  that  the  expression  for  k  is  a  polynomial  in  n  and  L  . 
Hence  the  algorithm  is  polynomial lv  bounded.  Note  a  Vo  that  the  expres¬ 
sion  for  k  is  not  particularly  tight,  as  a  much  tighter,  but  more  com¬ 
plicated  bound  would  result  if  the  exact  expression  for  the  contraction 
of  volumes  were  used,  as  well  as  the  exact  expression  for  the  volume  of 
the  initial  sphere.  To  obtain  this  bound,  we  would  seek  the  smallest 
k  such  that 


where  the  right-hand  side  is  a  better  bound  on  the  volume  of  the  solu¬ 
tion  space.  While  this  is  a  much  tighter  bound  than  derived  before, 
the  polynomial  boundedness  property  is  not  apparent. 

5.2  Is j olynomial  boundedness  desirable? 

An  algorithm  which  is  known  to  be  polynomialiy  bounded  may  or 
may  not  be  superior  to  one  which  is  not.  Polynomial  boundedness  is  a 
desirable  characteristic  of  an  algorithm,  but  may  not  accurately  depict 
performance  over  the  long  run. 
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Dantzig  [1]  compares  the  estimated  running  time  of  the  ellip¬ 
soidal  method  to  the  actual  running  time  of  a  sophisticated  implementa¬ 
tion  of  the  Simplex  Method  on  a  particular  case  study.  He  concludes 
that  the  Simplex  Method  wins  by  a  factor  of  30  minutes  to  50,000,000 
years!  This  is,  however,  computed  using  the  loose  upper  bound  and  con¬ 
sistently  setting  6=0  . 

McCall  [8]  studied  the  ellipsoidal  method  in  more  detail.  Among 
other  negative  results,  he  concluded  that  "Khachian's  algorithm  is  of 
no  practical  value  for  real-world  hP  problems." 

A  more  useful  statistic  for  measuring  performance  might  be  the 
expected  number  of  steps  required  to  find  a  feasible  solution  to  a  sys¬ 
tem  of  linear  inequalities.  Years  of  observing  the  Simplex  Method  at 
work  suggest  that  this  number  is  usually  less  than  4m  ,  where  m  is 
the  number  of  constraints  [1]. 

In  spite  of  criticism,  the  practical  import  of  this  method  has 
yet  to  be  evaluated.  Considerable  work  is  on-going.  As  of  February  6, 
1980,  Wolfe  [12]  had  compiled  a  list  of  23  research  references  relating 
to  the  new  algorithm,  and  this  list  is  surely  out-dated  by  now  (March 
25,  1980). 


6.  How  to  Solve  Linear  Programs 

Gacs  and  Lovasz  [2]  suggest  the  following  scheme  to  set  a  linear 
program  Ln  the  required  setting.  The  problem 


and  its  dual 


minimize  cx 

subject  to  Ax  >  b 
x  >  0 

maximize  ub 

subject  to  uA  <  c 
u  >  0 
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have  solutions  if  and  only  if  the  system 

Ax  >  b 
uA  <  c 
x  >  0 
u  >  0 
ub  >  cx 

has  a  solution.  The  latter  is  in  the  required  form. 

A  potential  difficulty  in  solving  the  latter  system  is  that  it 
generally  will  have  a  unique  solution.  Small  perturbations  will  yield 
approximate  solution  sets,  but  with  small  volumes. 

Another  variant,  employing  fewer  variables  and  fewer  constraints 

than  the  above  scheme  is  first  to  locate  a  point  feasible  to  P 

using  the  ellipsoidal  method.  Then,  continue  the  method  by  adding  the 

constraint  cx  <  cx^  to  the  constraints  Ax  ^  b  ,  x  >  0  .  Contracting 
ellipsoids  will  continue  to  encapsulate  that  portion  of  the  feasible  re¬ 
gion  which  contains  any  feasible  point  offering  better  values  to  cx 
than  the  current  one. 


TABLE  II 


CONVERGENCE  IN  EXAMPLE,  6*0 


Iteration  // 

xi 

X2 

Row  a 

bll 

b12  '  b21 

b22 

0 

0.000 

0.000 

(-1,0) 

100 

0 

100 

1 

3.333 

0.000 

(0,-1) 

44.44 

0 

133.3 

2 

3.333 

3.849 

(1,1) 

59.26 

0 

59.26 

3 

1.518 

2.035 

(1,1) 

52.67 

-26.34 

52.67 

4 

0.309 

0.825 

(-1,0) 

58.53 

-46.82 

58.53 

5 

2.859 

-1.215 

(0,-1) 

26.01 

-20.81 

44.74 

6 

1.822 

1.014 

feasible 

III.  Figure  7  illustrates  the  sequence  of  points  generated  by  both 
schemes,  while  Figures  8a  through  9g  illustrate  tbr  sequence  of  ellip¬ 
soids  generated  by  both  the  basic  and  the  "deep  cut11  schemes. 

It  is  interesting  to  note  that  the  latter  scheme  requires  one 
more  iteration  than  does  the  unrefined  method.  The  more  significant  de¬ 
crease  in  volumes  is  apparent  in  the  second  scheme,  however. 
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jective. 


*s  -  V#1 


